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This study is aimed at ensuring the validity of the Practical English 
Certification Test (PECT) of the Chung-nam Office of Education (COE) 
in Korea. Motivated by the demand for a developing localized English 
test to empower English learning in public education, the COE 
conducted the PECT for 38,544 students of elementary, middle and high 
schools in Chung-nam Province. The present study was designed to 
conduct the content and construct the validity of the PECT. Content 
validity was investigated by a critical review of researchers and 
educators in English education and proposed that the PECT is fulfilled 
with clarity in wording, relevance of items, formatting of items and 
clarity of the instructions. Likewise, in order to ensure the construct 
validity of the PECT, the present study sampled 148 students (i.e., 52 
elementary, 44 middle and 57 high school students) from a total of 
38,544 PECT takers and encouraged them to take the standardized 
English tests (i.e., PELT for elementary level and TOEIC® for 
secondary level). The findings reported that the PECT is substitutable 
for the standardized tests, PELT and TOEIC®. In short, the study 
suggests that conducting the localized English test, the PECT, seems 
worth trying in order to measure learners’ achievement accomplished by 
English learning in public education. 
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1 Introduction 

Motivated by the demand for encouraging public English education, the 
Chung-nam Office of Education (COE) developed a localized English test 
called the Practical English Certification Test (PECT). According to Chung 
(2013), the PECT appeals to the public with its unique feature of questions 
being derived from English textbooks, which is the staple medium of English 
instruction in public schools. Although several standardized English tests 
exist, in a sense, it is regarded that those tests are inadequate to measure 
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learners’ achievement of English learning in the public education system 
(Jung, 2013). While most schools around the world commonly use 
standardized tests to assess the level of students’ language use, there are still 
other types of tests administrated by departments within schools to measure 
how to improve language skills in each school (Akiyama, 2004). In the case 
of National English Ability Test (NEAT) in Korea, it is regarded as a type of 
test developed by the Korean Ministry of Education to assess and consider 
English ability only for Korean learners. According to the elucidation of 
NEAT (refer to http://www.neat.re.kr), one goal is to measure the 
achievement of English learning from each level of schools in Korea and it 
was designed to empower English learning through public education. In this 
light, the COE was encouraged to develop a localized English test for 
students in Chung-nam Province and developed the test with a group of 
English education professionals. In the process of developing the new test, 
the process of confirming the test’s validity is crucial, as its validity 
addresses whether or not a test truly measures what it was created to measure. 
According to Bachman (1990) and Bachman and Palmer (1996), test validity 
is defined as the meaningfulness and appropriateness of interpretations that is 
made on the basis of test scores. Alderson, Clapham and Wall (1995) and 
Choi and Kim (2012) mentioned that test validity consists of content validity 
reviewed by professional experts in the area of testing and face validity as 
described by test takers. However, there is some criticism about the content 
and face validity since they reflect the subjective point of view of reviewers. 
Thus the confirmation of construct validity is requested to compensate the 
validity with supporting evidence and it is mostly investigated by a statistical 
analysis such as correlation analysis (Lee, 2007; 2012). In this study, 
regarding the elucidation of the PECT and the importance of validity of the 
test, the study framed the research questions as follows. 

1) How is the quality of the Practical English Certification Test 
(PECT) of the COE for elementary, middle and high school students in 
Chung-nam Province? 

2) What is the evidence related to the use of the Practical English 
Certification Test (PECT) of the COE as a standardized test? 

2 Resources 

2.1 Practical English Certification Test (PECT) 

According to the elucidation of the PECT on the website of the COE and 
Chung (2013), the PECT is aimed to measure four English skills—listening, 
speaking (i.e., indirect way), reading and writing and pursues to resolve the 
imbalance in listening and reading oriented English tests that are commonly 
used to measure English ability in Korea (Jung, 2013). The COE announced 
that the PECT and the annual English Textbook Contest are their leading 
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English programs, since the English Textbook Contest has a reputation that 
its instructional objective is to encourage students to memorize English 
textbook content, and expects English learners to pay more attention to public 
education. Along with the success of the English textbook contest, the COE 
developed the PECT, which is aimed at containing topics and vocabulary 
from English textbooks, which are crucial instructional material in the 
English classrooms of elementary, middle and high schools in Chung-nam 
Province. 

The COE administrates the PECT for students in Chung-nam Province 
and provides three versions of the PECT tailored for elementary, middle and 
high school students. Each version of the test consists of two language parts: 
vocabulary-listening and reading-writing and two question types: sentence 
completion and multiple-choice type. Depending on the version of the test, 
the test time ranges from 40 to 50 minutes and the total number of questions 
ranges from 20 to 40. Table 1 shows more details of the PECT. 


Table 1. Details of PECT 


School 

Part 

Period 

Time 

Num. of 
Question 

Type of 
Question 

Elementary 

Vocabulary- 

Listening 

1st 

10:00-10:40 

(40min) 

20 

Completion & 
Multiple- 
choice 


Reading- 

Writing 

2nd 

11:00-11:40 

(40min) 

20 

Completion & 
Multiple- 
choice 

Middle 

Vocabulary- 

Listening 

1st 

10:00-10:45 

(45min) 

30 

Completion & 
Multiple- 
choice 


Reading- 

Writing 

2nd 

11:00-11:45 

(45min) 

30 

Completion & 
Multiple- 
choice 

High 

Vocabulary- 

Listening 

1st 

10:00-10:50 

(50min) 

40 

Completion & 
Multiple- 
choice 


Reading- 

Writing 

2nd 

11:00-11:50 

(50min) 

40 

Completion & 
Multiple- 
choice 


The COE promulgated the rubrics of the PECT to measure student’s 
performance on the test and are described as follows. 

- A score of 90 or above will get students Level 1 

- A score of 75 or above will get students Level 2 

- A score of 60 or above will get students Level 3 
(i.e. the highest score is 100) 

2.2 Standardized tests 
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To confirm the construct validity of the PECT, the study conducted 
standardized tests for a random sample of students. The standardized tests 
used in this study are the TOEIC® (Test of English for International 
Communication) by ETS (Educational Testing Service) and the PELT 
(Practical English Level Test) by the KFLEI (Korea Foreign Language 
Evaluation Institute) According to the official TOEIC® website 
(https://www.ets.org/ko/toeic), the TOEIC® is aimed to measure the business 
English competence of non-native speakers of English. The test consists of 
200 questions, which are designed to measure listening and reading 
comprehension. The PELT’s goal is to measure learners’ English literacy in 
vocational area and consists of 70 questions, which are designed to measure 
listening, reading, and writing competence (refer to http://www.pelt.or.kr/). 

3 Research methods 

3.1 Participants 

To confirm the content validity of the PECT, seven English education 
professionals were recruited, some of whom were professors in an English 
department and some of whom were English teachers of elementary, middle 
and high schools in Chung-nam Province. Table 2 illustrates these 
demographic variables of professionals. 


Table 2. Demographic Variables of Professionals 


Participants* 

Affiliation 

Education 
(Degree in Major) 

Teaching 

Experience) Y ears) 

1 

K University 

Ph. D. in English 
Education 

25 

2 

S University 

Ph. D. in English 
Education 

10 

3 

Y Elementary School 

BA in elementary 
education 

15 

4 

B Elementary School 

BA in elementary 
education 

10 

5 

B Middle School 

BA in English 
education 

25 

6 

W Middle School 

ME in English 
education 

10 

7 

S High school 

MA in TESOL 

8 


Participants: To secure anonymity, a numbering code was used. 
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To measure the construct validity of the PECT, a total of 38,544 students (i.e., 
30,387 elementary, 6,120 middle school and 2,057 high school students) 
were asked to take the PECT of the COE and received English certification of 
level 1, 2 or 3, respectively. Out of 38,544 test takers, the present study 
sampled 148 students (i.e., 52 elementary, 44 middle school and 57 high 
school students) and asked them to take the standardized English test (i.e., 
PELT for elementary and TOEIC® for secondary school students 
respectively). The participants were randomly sampled, so they consist of one 
heterogeneous group. Table 3 shows the PECT levels of the participants in 
this study. 


Table 3. Levels of PECT of Participants 


School 

Level* 

Results of PECT (total=148) 

Elementary 

1 

48 


2 

4 


3 

0 


Sub. 

52 

Middle 

1 

13 


2 

16 


3 

15 


Sub. 

44 

High 

1 

17 


2 

20 


3 

20 


Sub. 

57 

* Level: 1 is the highest score. 


3.2 Instruments 


To confirm the first research question, the study conducted the content 
validity test by the seven English education professionals, with the average 
inter-rater reliability reported as significant (Cronbach’s a=.89). It is 
assumed that they evaluated the content of PECT on the basis of their 
experience and knowledge of teaching English. In this study, the collected 
reviews were analyzed on the basis of the criteria of content validity as 
proposed by Fowler (2002) and they are as follows. (1) Clarity in wording; 
(2) Relevance of items; (3) Use of standard English; (4) Absence of biased 
words and phrases; (5) Formatting of items; (6) Clarity of instructions. For 
the second research question, the study conducted a comparison study 
between the PECT and two other standardized tests, the TOEIC® and the 
PELT, in order to confirm the construct validity of the PECT. The collected 
data were reviewed by correlation analysis, ANOVA and regression analysis. 
The statistical analysis was made via SPSS ver. 17.0 
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3.3 Gist of validation process 

In this study, the validity of the PECT was ensured by four stages: (1) COE 
administrated PECT to students in Chung-nam Province and assigned the 
PECT level from 1 to 3 for every test taker; (2) The content validity was 
assessed by the seven English education professionals; (3) The construct 
validity was assessed for 148 students, who were randomly sampled from the 
total of 38,544 PECT takers; (4) Collected data were analyzed and discussed 
in this study. 

4 Findings and discussion 

4.1 Results of content validity test 

To confirm content validity of PECT, seven English education professionals 
(i.e., 2 professors and 5 teachers) were asked to evaluate the test. Their 
reviews were analyzed on the basis of the content validity criteria as 
suggested by Fowler (2002) and the findings are summarized as follows. First, 
the PECT received positive feedback on ‘clarity in wording’, ‘relevance of 
the items’, ‘formatting of items’ and ‘clarity of instructions’. Second, 
comments on ‘use of Standard English’ and ‘absence of biased words and 
phrases’ were made differently depending on the background of reviewers. 
The professors evaluated that the PECT is dominated by American English, 
while teachers of elementary and secondary school assessed that the PECT 
uses English appropriately. Third, the English education professionals 
indicated that the PECT consists of appropriate content to measure learners’ 
practical English skills because it includes topics and vocabulary from 
textbooks mediated in elementary, middle and high schools. Fourth, English 
teachers in elementary, middle and high schools were asked to review the 
PECT on the basis of curricular validity and they reported that the PECT has 
a high relevance to the goals and objectives of the school curriculum and 
Korean national curriculum. 

4.2 Results of construct validity test 

4.2.1 PECT for elementary students 

To confirm the construct validity of the PECT for elementary students, a 
comparison was conducted between the PECT and the PELT (i.e., the 
standardized test for measuring practical English skills for the elementary 
level of students). The results of descriptive statistics of the PECT and the 
PELT are summarized in Table 4. 
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Table 4. The Results of Descriptive Statistics of PECT and PELT 



N 

Mean 

SD 

PECT 

50 

1.0660 

.23990 

PECT part 1 

50 

22.7800 

25.88064 

PECT part 2 

50 

18.9400 

1.40567 

PELT 

50 

156.2000 

26.37640 

PELT part 1 

50 

91.6400 

12.79471 

PELT part 2 

50 

64.5600 

16.16264 


Note: PECT part 1: Vocabulary + Listening/ part 2: Reading + Writing 


PELT part 1: Listening/ part 2: reading 

The statistical comparison of the two tests was conducted by correlation 
analysis and the result shows the significant correlation between the two tests 
(R=-.383, P<.01). Table 5 shows the results of a correlation analysis of the 
PECT and the PELT. 


Table 5. The Results of Correlation Analysis of PECT and PELT 




PECT 

PELT 

PECT 

Pearson Correlation 

1 

-.383** 


Sig. (2-tailed) 


.006 


N 

50 

50 

PELT 

Pearson Correlation 

-383** 

1 


Sig. (2-tailed) 

.006 



N 

50 

50 


In addition, the study proposed that further analysis to investigate the effect 
of the PECT on the result of the PELT, thus a regression analysis was 
conducted. As a result, the dependent variable, the PECT, shows a significant 
effect on the results of the PELT (T—2.868, P<.01). Table 6 shows the 
results of regression analysis of the PECT and the PELT. 


Table 6. The Results of Regression Analysis of PECT and PELT 



Unstandardized 

Coefficients 

Standardized 

Coefficients 



Model 

B 

Std. Error 

Beta 

t 

Sig. 

1 (constant) 

1.603 

.192 


8.347 

.000 

PELT 

-.003 

.001 

-.383 

-2.868 

.006 


a. Dependent Variable: PECT 


4.2 PECT for secondary students 

To confirm the construct validity of the PECT with middle and high school 
students, the comparison between the PECT and the TOEIC® was conducted 
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by ANOVA. The statistical findings suggest a significant correlation 


(F=20.540, P<.01) between the PECT and the TOEIC®. Table 7 shows the 
results of ANOVA. 

Table 7. The Results of ANOVA of PECT and TOEIC® 

Model 

Sum of 
Squares 

df Mean 

Square 

F Sig. 

1 Regression 

337138.4 

82 

1 337138.482 

20.540 .000(a) 

Residual 

1608560 

518. 

98 16413.883 


TOEIC® 

1945699 

99 



000 


a. Predictors: (Constant), PECT 

b. Dependent Variable: TOEIC® 

The investigation by regression analysis was also made to examine the effect 
of the PECT on the result of the TOEIC® and the study found a significant 
effect of the PECT on the TOEIC® (T=4.532, P<.01). Table 8 shows more 
details. 


Table 8. The Results of Regression Analysis of PECT and Mock TOEIC® 



Unstandardized 

Standardized 




Coefficients 

Coefficients 



Model 

B 

Std. Error 

Beta 

t 

Sig. 

1 (constant) 

617.120 

35.906 


17.187 

.000 

TOEIC® 

-73.439 

16.204 

-.416 

-4.532 

.000 


a. dependent variable: PECT 


4.3 Discussion 

The present study conducted a validity test of the PECT of the COE and 
found that the PECT is substitutable for a standardized test for measuring 
English competence of the learners of Chung-nam Province. Further 
discussion is as follows. 

First, the content validity of the PECT was confirmed by the 
assessment of the seven English education professionals and revealed that the 
PECT contains clarity in wording, relevance of items, formatting of items 
and clarity of the instructions, which are suggested in the validity criteria of 
Fowler (2002). Significantly, different opinions were recognized in the 
reviews, which are related to the use of Standard English and the absence of 
biased words and phrases. It is assumed that college professors prefer World 
Englishes to American English for language use, while school teachers 
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respect American English as the only English to be targeted during English 
instruction. This is supported by the previous research about English 
textbook in EFL settings (Lee, 2009; Matsuda, 2002). English textbooks in 
Korea are filled with US culture and language, while English textbooks in 
Japan show a strong inner-circle orientation in their choice of linguistic 
samples. Since PECT is aimed at complying topics and vocabulary with 
English textbooks, the dominant culture and language of the textbooks may 
reflect the test materials. Since English textbooks are regarded as crucial 
teaching materials in most EFL classrooms (Sercu, 2000), the featured 
culture and language of the English textbook was an influence in the 
developing language preference of both the teachers and students (Chang, 
2011). Therefore it is interpretable that English teachers may show a strong 
tie to American English, which dominates the English textbooks throughout 
Korea (Chang, 2007, 2013; Huh, 2013; Lee, 2007, 2012; Ok, 2014). Thus, 
whether teachers use Standard English or World Englishes is less persuasive 
than college professors who tend to use World Englishes more in ELT 
(Byram, 1997; Chang, 2012, 2013; Lee, 2007, 2012; Mukminatien, 2012). 

Second, the present study proposed that PECT is suitable as a 
standardized English test to assess the English competence of learners in 
Korea. The questions of the PECT were created by referring to the topics and 
vocabulary of the English textbooks, thus they are eligible to measure the 
English achievement from learning in public education. The previous study 
by Mbali and Douglas (2012) insisted that the goal of language test is to 
measure English learning from schools and they used the localized English 
oral test for 6th graders of a South African school. O’Malley and Valdez 
(1996) defined authentic assessment as “the multiple forms of assessment 
that reflect student learning, achievement, motivation and attitudes on 
instmctionally-relevant classroom activity” (p.4). Accordingly, if the 
localized test is significantly correlated with the standardized tests, it is seen 
as a positive signal to alternate the current standardized test with the localized 
test. According to Brown (1996) and Brown and Abeywickrama (2010), 
achievement test have as their primary goal determining the acquisition of 
course objectives at the end of a period of instruction, thus the devotion for 
developing the localized English competence test is seen to be worth a try for 
English learners in Korea. 

5 Conclusion 

The present study is designed to ensure the validity of the language test for 
EFL learners of a local region in Korea. It originated from the national 
educational policy to empower English learning in public education, thus a 
transformation of the motivation of students learning English in public 
education was requested. If students are assessed by what they have learned 
in schools, they will exert themselves to learn English according to their 
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English instruction in schools. Thus the test will meet the goals of Korean 
educational policy, which is to provide an educational balance to all students. 
For this reason, the COE and other education offices in Korea are encouraged 
to develop localized English tests. As a result, CEO developed the PECT and 
the present study found that the PECT is substitutable for the standardized 
test; however, the present study has a limitation to conclude that the PECT is 
reliable as an alternative standardized test in general, because the PECT was 
conducted only for students in Chungnam province and omitted to conduct 
face validity conducted by test takers. For further study, it is strongly 
suggested that future research design is formed to investigate on the aspects 
of the test takers. 
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